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Abstract Body 


Background / Context: 

Why are some schools able to thrive and produce strong student outcomes, while others 
struggle? To answer this question, researchers have surveyed teachers and students in Chicago 
Public Schools (CPS) since the 1990s, asking them about their schools’ organizational climate 
and practices. From these data, researchers at the Consortium on Chicago School Research 
(CCSR) concluded that five essential supports of the schools’ organization facilitate engaging 
instruction and learning: effective leadership, collaborative teachers, involved families, 
supportive environments, and ambitious instruction. Schools strong in these practice domains 
were much more likely than schools weak in these areas to see improvements in student 
outcomes, including attendance and learning gains (Bryk, Sebring, Allensworth, Luppescu, & 
Easton, 2010). 

Research on school climate and school organization, including that of CCSR, influenced 
Illinois legislators in 201 1 to mandate the collection of data on teacher and student perceptions of 
schools’ instructional environments to provide feedback to principals. The state turned to 
CCSR’s 5 Essentials survey instrument, and it was fielded statewide for the first time in spring of 
2013. Students (grades 6-12) and teachers took surveys measuring the essential supports in their 
schools, allowing us to examine the extent to which schools across Illinois are strong or weak in 
these essential supports and better understand how the supports function in different community 
contexts across Illinois. 

Research Questions: 

Illinois took measures of essential supports tested and refined in one context (CPS) and 
required its administration statewide. Using this statewide data, we address two overarching 
research questions: 

(1) How does strength and weakness on the five essential supports vary according to urbanicity, 
enrollment size of school, and socioeconomic characteristics of school communities? 

(2) Are the five essential supports related to student outcomes including attendance rates, test 
scores, and graduation rates? 

Setting: 

Surveys measuring the essential supports were administered to students and teachers in 
nearly all public schools in Illinois in the spring of 2013. 

Population / Participants / Subjects: 

Students in grades 6-12 and all full-time classroom teachers in public schools in Illinois 
were eligible to take the survey. Out of 152,462 teachers in Illinois, 104,270 (68 percent) took 
the survey, and out of 1,101,025 students in grades 6-12 in Illinois, 750,329 (68 percent) 
completed the survey. All told, usable data was collected from 3,684 Illinois schools (85 
percent). 

Intervention / Program / Practice: 

Survey data were collected via the web. School staff facilitated data collection among 
students and monitored response rates for both students and teachers. The surveys took on 
average 20 minutes to complete. 
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Research Design: 

Cross-sectional survey data were linked to official data on school and community 
demographics and student outcomes, aggregated at the school level. 

Data Collection and Analysis: 

The Essential Supports 

The 5Essential survey items are submitted to Rasch measurement models to create 22 
different measures, which in turn are averaged to constitute the five essential supports. We also 
constructed a measure of the “average essential” based on the mean of the five essential supports. 
See Figure 1 for a listing of each measure and their corresponding essential support. 

(please insert figure 1 here) 

Community Contexts Shaping the Essential Supports 

For community contexts, we used measures from the U.S. Census Bureau and Common 
Core of Data (CCD). For urbanicity, we distinguished between CPS schools', schools outside 
Chicago in Census-designated urban areas (territories with at least 50,000 residents that are 
principal cities in core-based statistical areas), suburban areas (territories with at least 50,000 
residents that are not principal cities), towns (territories with 2,500-50,000 residents), and rural 
areas (territories with fewer than 2,500 residents). School enrollment size is the average of 
student membership as recorded in the 2010-11 and 2011-2012 CCD. Socioeconomic 
disadvantage is a standardized scale of percent of students eligible for free or reduced-price 
lunches (averaging rates in the 2010-11 and 2011-12 school years), the poverty rate in the 
school’s Census tract, and the percent of jobless males aged 16 or older in the school’s Census 
tract (Census tract variables come from the 2007-11 American Community Survey). 

Student Outcomes 

We examined a variety of student outcomes measured for a three year period spanning 
the 2010-11, 2011-12, and 2012-13 school years. For both the primary and secondary grades, we 
measured schools’ average attendance rates and schools’ change in attendance rates. Both of 
these measures were calculated over the three year period. 

For the elementary and middle grades, we had access to the Illinois Standards 
Achievement Test (IS AT) math and reading test scores (a test given to all Illinois students in 
grades 3-8) of all Illinois students for the springs of 201 1, 2012, and 2013. We performed 
multilevel modeling on this data to measure each school’s average IS AT growth rate, as well 
as their average IS AT scores. 

For the high school grades, we took schools’ average graduation rates and average ACT 
scores (in Illinois, all high school students in their junior year take the ACT test). We tried 
estimating change in graduation rates and ACT scores but the reliabilities of schools’ trajectories 
were extremely low. 

Control Variables 

We controlled for a variety of factors: urbanicity (defined above), enrollment size, and 
socioeconomic disadvantage (defined above), socioeconomic advantage (a standardized scale of 
the years of schooling of adults and the percent of employed civilians in managerial and 
professional occupations in the school’s Census tract, according to the 2007-2011 ACS), school 
racial composition, charter status, and grade configuration. When average ISAT growth rate is 
the outcome, we also controlled for schools’ initial value (their estimated average ISAT score in 
2010 - 2011 ). 
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When we examine elementary and middle schools, we used school district fixed effects; 
when we examine high schools, we could not do this because few districts have multiple high 
schools. Instead, for high school outcomes we controlled for the following district 
characteristics: per pupil expenditures, number of schools in the district, and district-level 
enrollment, socioeconomic advantage, socioeconomic disadvantage, and racial composition. 
Analyses 

To understand the links between community context and the Essential Supports, we did 
basic descriptive analyses without controlling for any covariates. To understand the association 
between the Essential Supports and student outcomes, we performed school-level regressions. 
When we analyzed high school outcomes, we perfonned the regressions separately for CPS 
schools and schools in the rest of Illinois. When we analyzed outcomes for elementary and 
middle schools, we performed the regressions separately for CPS schools with the K-8 and 6-8 
grade configurations; non-CPS schools with K-8 and 6-8 grade configurations; and schools with 
a K-5 grade configuration. We had to separate the K-5 schools because their students were not 
surveyed and thus they had incomplete information on the Essential Supports. 

Findings / Results: 

Associations between Community Context and the Essential Supports 

We found substantial differences among schools in the degree to which students and 
teachers report strength in the essential supports. A higher proportion of urban and suburban 
schools are strong in supportive environment and ambitious instruction compared with schools in 
towns and rural areas. This advantage is particularly pronounced in CPS schools. Schools 
serving students with socioeconomic disadvantages are less likely to be strong in the essential 
supports, compared to schools serving more affluent students. We also found differences among 
schools based on their size. Smaller schools are more likely to have strong essentials than are 
larger schools, although the pattern is not as clear at the high school level. 

Associations between the Essential Supports and Student Outcomes 

Standardized coefficients showing the relationships between the essential supports and 
student outcomes are presented in Table 1. For the most part, we see positive and significant 
associations, indicating that schools stronger in the essential supports tend to have better student 
outcomes. However, by conventional standards these associations tend to be weak, hovering 
around a standardized coefficient of . 10. The main exception is for high school outcomes in CPS 
schools, where the coefficients tend to be large, ranging from 0.2 to 0.4. 

(please insert Table 1 here) 

Lipsey et al. (2012) have argued that conventional guidelines for effect sizes are not 
useful in education research. We thus provide a benchmark in Table 2, the standardized effect of 
socioeconomic disadvantage on each outcome. Using this benchmark, the associations between 
the essential supports and ISAT growth rates tend to be fairly substantial in K-8 and 6-8 schools 
throughout Illinois, being at least half of the association between socioeconomic disadvantage 
and ISAT growth rates. However, even using this benchmark the essential supports have small 
associations with average ISAT scores and elementary school attendance rates. 

(please insert Table 2 here) 

The essential support with the largest associations is Supportive Environment, following 
by Involved Families and Ambitious Instruction. This supports the essential supports 
framework, which posits that Supportive Environment and Ambitious Instruction are the closest 
(in a causal sense) to student outcomes. 


SREE Spring 2016 Conference Abstract Template 


3 



The associations tend to be weak for K-5 schools. One possible explanation for the small 
associations — even for average ISAT growth rates — is that K-5 schools are missing two 
essential supports, Supportive Environments and Ambitious Instructions (because these essential 
rely on student reports, which were only collected for students in grades 6-12). In future work 
the Consortium of Chicago School Research will be investigating the utility of collecting student 
reports in grades 4-5, allowing us to test this possibility. 

The fact that the essential supports are more highly associated with student outcomes in 
CPS high schools than in the rest of the state is noteworthy. There are three potential reasons for 
why this is. First, CPS teachers and school officials have had greater exposure to the essential 
supports framework, and they may be more truthful in their reports of the implementation of the 
essential supports. Second, the 5Essential surveys were designed for the Chicago context, and 
may be missing relevant aspects of organizational climate in schools outside of large urban 
contexts (however, in other analyses not presented here we found that the essential supports had 
larger associations in CPS than in non-Chicago urban contexts in Illinois). Third, the 
associations in Chicago may reflect particularly strong selection processes; high-perfonning 
students may be drawn to schools that are already strong in the essential supports. Chicago’s 
selective enrollment high schools are of particular concern here; students who enroll in these 
schools are selected on the basis of their test scores and grades. Indeed, removing these schools 
from the analysis does result in the associations in CPS declining substantially (see Table 2); but 
they are still stronger than the associations outside of Chicago and they are a substantial fraction 
of the association with socioeconomic disadvantage. 

Conclusions: 

This study had two goals. First, to document disparities in essential supports across 
schools in the entire state. We show that students in rural schools and socioeconomically 
disadvantaged schools are less likely to experience strong essential supports. The fact that 
students attending socioeconomically disadvantaged schools have much less access to the 
essential supports raises substantial equity concerns. Indeed, these students may be most in need 
of schools that are especially strong. It is important to note that locational inequalities are 
strongest in ambitious instruction, the essential that is theorized to have the most direct influence 
on student outcomes. 

The second goal is examine the predictive validity of the 5Essential survey outside of 
Chicago. Overall, the essential supports have positive albeit small associations with student 
outcomes. However, some of these associations, particularly test score growth in elementary 
schools, are sizable when they are compared to the associations between student outcomes and 
socioeconomic disadvantage. Because of data limitations, we cannot prove if the essential 
supports, as measured by the 5Essentials survey, actually influence student outcomes in Illinois. 
Rather, this is a first step towards understanding what the essentials framework has to offer 
schools outside of Chicago. The essential supports offer useful infonnation about how 
elementary and middle schools (particularly K-8 and 6-8 schools) are organized in Illinois. The 
weak associations in high schools outside of CPS indicate there may be better ways to capture 
the essential supports in those contexts. 

It is our hope that by collecting and accumulating data for Illinois schools in the future, 
researchers will gain a thorough understanding of how to measure school climate in various 
contexts and provide schools and districts with infonnation about how school climate matters (or 
does not matter) for student outcomes. 
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Appendix B. Tables and Figures 

Figure 1 • 5Essentials Survey Measures 


Ambitious Instruction 

Involved Families 

- Course Clarity (S) 

- Human & Social Resources in the Community (S) 

- English Instruction (S) 

- Outreach to Parents (T) 

- Math Instruction (S) 

- Teacher-Parent Trust (T) 

- Quality of Student Discussion (T) 



Supportive Environment 

Effecive Leaders 

- Peer Support for Academic Work (S) 

- Teacher Influence (T) 

- Academic Personalism (S) 

- Principal Instructional Leadership (T) 

- Academic Press (S; elem only) 

- Program Coherence (T) 

- Safety (S) 

- Teacher-Principal Trust (T) 

- Student-Teacher Trust (S) 


- School-wide Future Orientation (S; HS only) 

Collaborative Teachers 

- Expectations for Post-Secondard Education (S; HS on 

- Collective Responsibility (T) 


- Quality Professional Development (T) 

(S) - Student measure; (T) - Teacher measure 

- School Commitment (T) 


- Teacher-Teacher Trust (T) 
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Table 1: Fully Standardized Regression Coefficients (“betas”) Showing Associations between Essential Supports and 
School Outcomes 




Average 

Essentials 

(beta) 

Effective 

Leaders 

(beta) 

Collaborative 

Teachers 

(beta) 

Involved 

Families 

(beta) 

Supportive 

Environment 

(beta) 

Ambitious 

Instruction 

(beta) 

Effect of 
Socioeconomic 
Disadvantage 
(beta) 

ISAT Math 

Gains 

Non-CPS 

.10* 

0.02 

0.05 

0.09| 

0.13* 

0.10* 

-0.17* 

CPS 

0.12* 

0.06f 

0.10* 

0.03 

0.10* 

0.14* 

K-5 

0.06f 

0.05 

0.04 

0.09* 



ISAT Reading 
Gains 

Non-CPS 

0.15* 

0.04 

0.06f 

0.20* 

0.20* 

0.20* 

-0.14* 

CPS 

0.07* 

0.05 

0.05f 

0.02 

0.04t 

0.09* 

K-5 

0.04 

0.03 

0.02 

0.07| 



ISAT Math Level 

Non-CPS 

0.12* 

0.06* 

0.11* 

0.14* 

0.06* 

0.05* 

-0.58* 

CPS 

0.07* 

0.00 

0.07* 

0.08f 

0.06* 

0.07* 

K-5 

0.12* 

0.09* 

0.11* 

0.15* 



ISAT Reading 
Level 

Non-CPS 

0.11* 

0.04* 

0.09* 

0.11* 

0.07* 

0.05* 

-0.66* 

CPS 

0.07* 

0.00 

0.07* 

0.10* 

0.05* 

0.07* 

K-5 

0.12* 

0.10* 

0.12* 

0.16* 



Attendance Rate 
Change 
(Elementary 
Schools) 

Non-CPS 

0.07* 

0.05f 

0.05t 

0.08f 

0.04 

0.04 

0.04f 

CPS 

0.19* 

0.11* 

0.13* 

0.18* 

0.15* 

0.18* 

K-5 

0.04 

0.03 

0.03 

0.06* 



Attendance Rate 
Level 

(Elementary 

Schools) 

Non-CPS 

0.09* 

0.03 

0.08* 

0.11* 

0.05t 

0.04 

-0.43* 

CPS 

0.03 

-0.05 

0.04 

0.01 

0.02 

0.07* 

K-5 

0.07* 

0.06* 

0.08* 

0.09* 



Attendance Rate 
Change (High 
Schools) 

Non-CPS 

0.08* 

0.03 f 

0.05* 

0.09* 

0.10* 

0.08* 

-0.03 

CPS 

0.22* 

0.16* 

0.21* 

0.30* 

0.17* 

0.15* 

Attendance Rate 
Level (High 
Schools) 

Non-CPS 

0.07* 

0.01 

0.06* 

0.09* 

0.11* 

0.07* 

-.44* 

CPS 

0.42* 

0.20* 

0.31* 

0.43* 

0.51* 

0.45* 

Average ACT 
Score Level 

Non-CPS 

0.10* 

0.02 

0.07* 

0.10* 

0.18* 

0.10* 

-0.80* 

CPS 

0.36* 

0.16* 

0.23* 

0.32* 

0.47* 

0.41* 

Graduation Rate 
Level 

Non-CPS 

0.10* 

0.03 

0.06* 

0.13* 

0.16* 

0.07* 

-0.48* 

CPS 

0.30* 

0.14 

0.22* 

0.27* 

0.36* 

0.34* 


\* p < .05; | p < .10 
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Table 2 • Comparing Associations of Essential Supports With High School Outcomes, With and 
Without Selective Enrollment High Schools in CPS 


Outcome 

Standardized 

Coefficent, 

non-CPS 

Standardized 
Coefficent, 
CPS (all 
high 
schools) 

Standardized 

Coefficent, 

CPS, no 

selective 

enrollment 

schools 

Standardized 

Coefficent, 

SCON, 

statewide 

ACT 

Scores 

.10 

.36 

.19 

-.80 

Grad Rates 

.10 

.30 

.22 

-.48 

Change in 

Attendance 

Rates 

.08 

.22 

.26 

-.03 

Attendance 

Rates 

.07 

.42 

.35 

-.44 
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